Potential-Based Shaping and Q-Value Initialization are Equivalent

Potential-Based Shaping and Q-Value Initialization are Equivalent

Shaping has proven to be a powerful but precarious means of improving reinforcement learning performance. Ng, Harada, and Russell (1999) proposed the potential-based shaping algorithm for adding shaping rewards in a way that guarantees the learner will learn optimal behavior. In this note, we prove certain similarities between this shaping algorithm and the initialization step required for seve...

Dynamic potential-based reward shaping

Potential-based reward shaping can significantly improve the time needed to learn an optimal policy and, in multiagent systems, the performance of the final joint-policy. It has been proven to not alter the optimal policy of an agent learning alone or the Nash equilibria of multiple agents learning together. However, a limitation of existing proofs is the assumption that the potential of a stat...

q-Markov covariance equivalent realizations

Potential-based Shaping in Model-based Reinforcement Learning

Potential-based shaping was designed as a way of introducing background knowledge into model-free reinforcement-learning algorithms. By identifying states that are likely to have high value, this approach can decrease experience complexity—the number of trials needed to find near-optimal behavior. An orthogonal way of decreasing experience complexity is to use a model-based learning approach, b...

Being right on Q: shaping eukaryotic evolution

Reactive oxygen species (ROS) formation by mitochondria is an incompletely understood eukaryotic process. I proposed a kinetic model [BioEssays (2011) 33: , 88-94] in which the ratio between electrons entering the respiratory chain via FADH2 or NADH (the F/N ratio) is a crucial determinant of ROS formation. During glucose breakdown, the ratio is low, while during fatty acid breakdown, the ratio...